Skip to content

Conversation

yamt
Copy link
Collaborator

@yamt yamt commented Jul 28, 2025

for wasi_ephemeral_nn,

  • implement u8 input

  • stop dealing with quantization.

    • wasi-nn doesn't have a concept of quantization or pre/post-processing. i can't think of any ways to make the backend perform zero-point/scale processing without risking to break other applications.

    • there seems to be applications which just use u8 inputs/outputs for a quantized model. (see [1] for an example.) for certain kinds of inputs/outputs, it usually just works.

this commit keeps the legacy wasi_nn logic intact for now.

tested with [1] with [2] applied.

WAMR with this patch:

Read graph weights, size in bytes: 3561598
[wasi_nn.c:297 WARNING] load_by_name_with_config() not found
[wasi_nn_tensorflowlite.cpp:272 WARNING] Default encoding is CPU.
Loaded graph into wasi-nn with ID: Graph#0
Read input tensor, size in bytes: 150528
   1.) [166](198)Aix galericulata
   2.) [34](1)Gallus gallus domesticus
   3.) [158](1)Coccothraustes coccothraustes
   4.) [778](1)Sitta europaea
   5.) [819](1)Anas platyrhynchos

wasmedge:

Read graph weights, size in bytes: 3561598
Loaded graph into wasi-nn with ID: Graph#0
Read input tensor, size in bytes: 150528
   1.) [166](198)Aix galericulata
   2.) [34](1)Gallus gallus domesticus
   3.) [158](1)Coccothraustes coccothraustes
   4.) [778](1)Sitta europaea
   5.) [819](1)Anas platyrhynchos

and "Aix galericulata" seems like a reasonable classification of the image to my eyes.

[1] https://github.com/second-state/WasmEdge-WASINN-examples/tree/67f174bab59d98c1b52f7367ec0928701dc998f9/tflite-birds_v1-image

[2] second-state/WasmEdge-WASINN-examples#204

Related:
#3555 #2611

for wasi_ephemeral_nn,

* implement u8 input

* stop dealing with quantization.

  * wasi-nn doesn't have a concept of quantization or pre/post-processing.
    i can't think of any ways to make the backend perform zero-point/scale
    processing without risking to break other applications.

  * there seems to be applications which just use u8 inputs/outputs for
    a quantized model. (see [1] for an example.)
    for certain kinds of inputs/outputs, it usually just works.

this commit keeps the legacy wasi_nn logic intact for now.

tested with [1] with [2] applied.

WAMR with this patch:
```
Read graph weights, size in bytes: 3561598
[wasi_nn.c:297 WARNING] load_by_name_with_config() not found
[wasi_nn_tensorflowlite.cpp:272 WARNING] Default encoding is CPU.
Loaded graph into wasi-nn with ID: Graph#0
Read input tensor, size in bytes: 150528
   1.) [166](198)Aix galericulata
   2.) [34](1)Gallus gallus domesticus
   3.) [158](1)Coccothraustes coccothraustes
   4.) [778](1)Sitta europaea
   5.) [819](1)Anas platyrhynchos
```

wasmedge:
```
Read graph weights, size in bytes: 3561598
Loaded graph into wasi-nn with ID: Graph#0
Read input tensor, size in bytes: 150528
   1.) [166](198)Aix galericulata
   2.) [34](1)Gallus gallus domesticus
   3.) [158](1)Coccothraustes coccothraustes
   4.) [778](1)Sitta europaea
   5.) [819](1)Anas platyrhynchos
```

and "Aix galericulata" seems like a reasonable classification
of the image to my eyes.

[1] https://github.com/second-state/WasmEdge-WASINN-examples/tree/67f174bab59d98c1b52f7367ec0928701dc998f9/tflite-birds_v1-image

[2] second-state/WasmEdge-WASINN-examples#204

Related:
bytecodealliance#3555
bytecodealliance#2611
@yamt yamt added wasi-nn breaking-change Determine if this PR introduces breaking changes. It will be used by scripts to classify PRs. labels Jul 28, 2025
@yamt
Copy link
Collaborator Author

yamt commented Jul 28, 2025

@TianlongLiang TianlongLiang reopened this Aug 1, 2025
@lum1n0us lum1n0us merged commit 29d465b into bytecodealliance:main Aug 1, 2025
517 of 522 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking-change Determine if this PR introduces breaking changes. It will be used by scripts to classify PRs. wasi-nn
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants